Papers with large-scale vision-language datasets
VEIL: Vetting Extracted Image Labels from In-the-Wild Captions for Weakly-Supervised Object Detection (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing methods to “vet” labels from noisy captions for weakly-supervised object detection are limited for object detection. |
| Approach: | They propose a technique to “vet” labels extracted from noisy captions and use them for weakly-supervised object detection without any bounding boxes. |
| Outcome: | The proposed method improves WSOD without label vetting by 30% on five datasets. |
Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings (2026.findings-acl)
Copied to clipboard
Aakriti Agrawal, Gouthaman KV, Rohith Aralikatti, Gauri Jagatap, Jiaxin Yuan, Sarvesh Baskar, Vijay Kamarshi, Andrea Fanelli, Furong Huang
| Challenge: | Hallucinations in Large Vision-Language Models (LVLMs) are a persistent challenge, stemming from inadequate integration of visual information during multimodal reasoning. |
| Approach: | They propose a visual feature incorporation method that encourages the model to learn visually-informed textual embeddings distinct from those of the base LLM and promotes a more balanced attention distribution. |
| Outcome: | The proposed method significantly reduces hallucinations and fosters more balanced multimodal reasoning. |